Basis Pursuit

The Time-Frequency and Time-Scale communities have recently developed a large number of overcomplete waveform dictionaries { stationary wavelets, wavelet packets, cosine packets, chirplets, and warplets, to name a few. Decomposition into overcomplete systems is not unique, and several methods for decomposition have been proposed, including the Method of Frames, Matching Pursuit and, for special dictionaries, the Best Orthogonal Basis method. Basis Pursuit is a principle for decomposing a signal into an \optimal" superposition of dictionary elements { where optimal means having the smallest l1 norm of coe cients among all such decompositions. We give examples exhibiting several advantages over the Method of Frames, Matching Pursuit and Best Orthogonal Basis, including better sparsity and super-resolution. Basis Pursuit has interesting relations to ideas in areas as diverse as ill-posed problems, abstract harmonic analysis, total variation de-noising, and multi-scale edge de-noising. Basis Pursuit in highly overcomplete dictionaries leads to large-scale optimization problems. With signals of length 8192 and a certain wavelet packet dictionary, one gets an equivalent linear program of size 8192 by 212,992. Such problems can be attacked successfully only because of recent advances in linear programming by interior point methods. We obtain reasonable success with a primal-dual logarithmic barrier method and conjugate gradient solver. iv Acknowledgments First, I would like to express my deep gratitude to my advisor Professor David Donoho for guiding me through the past four years of study at Stanford, for his willingness to share his ideas in research problems, for the time and energy he put in developing my research skills and advising my thesis work, for his patience in correcting my English writing, and for the kindness and generosity he showed not only to me, but also to his other students. Second, I would like to thank Professor Iain Johnstone for giving me advice on my research and job searching, for arranging me to attend the Wavelet conference at ANU Australia in June 1995, and for writing letters of recommendation to IBM for me.I would also like to thank Professor Michael Saunders, for kindly providing advice, software and wisdom about large-scale linear programming and least squares. Special thanks to Helen Tombropoulos, for kindly helping me with my English writing, and for bringing fresh owers from her garden to Sequoia almost everyday! Last, I would like thank my friends in the Chinese Christian Fellowship at Stanford. Their constant prayers, support and encouragement have made my days at Stanford pleasant and unforgettable. v